Optical proximity sensor and associated user interface

ABSTRACT

A sensor, including light emitters projecting directed light beams, light detectors interleaved with the light emitters, lenses, each lens oriented relative to a respective one of the light detectors such that the light detector receives maximum intensity when light enters the lens at an angle b, whereby, for each emitter E, there exist corresponding target positions p(E, D) along the path of the light from emitter E, at which an object located at any of the target positions reflects the light projected by emitter E towards a respective one of detectors D at angle b, and a processor storing a reflection value R(E, D) for each co-activated emitter-detector pair (E, D), based on an amount of light reflected by an object located at p(E, D) and detected by detector D, and calculating a location of an object based on the reflection values and target positions.

CROSS REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/000,815 entitled OPTICAL PROXIMITY SENSOR AND ASSOCIATED USERINTERFACE and filed on Jan. 19, 2016 by inventors Thomas Eriksson,Alexander Jubner, Rozita Teymourzadeh, Håkan Sven Erik Andersson, PerRosengren, Xiatao Wang, Stefan Holmgren, Gunnar Martin Fröjdh, SimonFellin, Jan Tomas Hartman, Oscar Sverud, Sangtaek Kim, Rasmus Dahl-Örn,Richard Berglind, Karl Erik Patrik Nordström, Lars Sparf, ErikRosengren, John Karlsson, Remo Behdasht, Robin Kjell Åman, Joseph Shain,Oskar Hagberg and Joel Rozada. U.S. patent application Ser. No.15/000,815 claims priority benefit from:

-   -   U.S. Provisional Application No. 62/107,536 entitled OPTICAL        PROXIMITY SENSORS and filed on Jan. 26, 2015 by inventors Stefan        Holmgren, Oscar Sverud, Sairam Iyer, Richard Berglind, Karl Erik        Patrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren,        John Karlsson, Thomas Eriksson, Alexander Jubner, Remo Behdasht,        Simon Fellin, Robin Kjell Åman and Joseph Shain;    -   U.S. Provisional Application No. 62/197,813 entitled OPTICAL        PROXIMITY SENSOR and filed on Jul. 28, 2015 by inventors Rozita        Teymourzadeh, Håkan Sven Erik Andersson, Per Rosengren, Xiatao        Wang, Stefan Holmgren, Gunnar Martin Fröjdh and Simon Fellin;        and    -   U.S. Provisional Application No. 62/266,011 entitled OPTICAL        PROXIMITY SENSOR and filed on Dec. 11, 2015 by inventors Thomas        Eriksson, Alexander Jubner, Rozita Teymourzadeh, Håkan Sven Erik        Andersson, Per Rosengren, Xiatao Wang, Stefan Holmgren, Gunnar        Martin Fröjdh, Simon Fellin and Jan Tomas Hartman.

U.S. patent application Ser. No. 15/000,815 is a continuation-in-part ofU.S. patent application Ser. No. 14/630,737, entitled LIGHT-BASEDPROXIMITY DETECTION SYSTEM AND USER INTERFACE and filed on Feb. 25, 2015by by inventors Thomas Eriksson and Stefan Holmgren.

U.S. patent application Ser. No. 14/630,737 is a continuation of U.S.patent application Ser. No. 14/140,635, now U.S. Pat. No. 9,001,087,entitled LIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE andfiled on Dec. 26, 2013 by inventors Thomas Eriksson and Stefan Holmgren.

U.S. patent application Ser. No. 14/140,635 is a continuation of U.S.patent application Ser. No. 13/732,456, now U.S. Pat. No. 8,643,628,entitled LIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE andfiled on Jan. 2, 2013 by inventors Thomas Eriksson and Stefan Holmgren.

U.S. patent application Ser. No. 13/732,456 claims priority benefit ofU.S. Provisional Patent Application Ser. No. 61/713,546, entitledLIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE and filed onOct. 14, 2012 by inventor Stefan Holmgren.

U.S. patent application Ser. No. 15/000,815 is a continuation-in-part ofU.S. patent application Ser. No. 14/726,533, now U.S. Pat. No.9,679,601, entitled OPTICAL TOUCH SCREENS and filed on May 31, 2015 byinventors Robert Pettersson, Per Rosengren, Erik Rosengren, StefanHolmgren, Lars Sparf, Richard Berglind, Thomas Eriksson, Karl ErikPatrik Nordström, Gunnar Martin Fröjdh, Xiatao Wang and Remo Behdasht.

U.S. patent application Ser. No. 14/726,533 is a continuation of U.S.patent application Ser. No. 14/311,366, now U.S. Pat. No. 9,063,614,entitled OPTICAL TOUCH SCREENS and filed on Jun. 23, 2014 by inventorsRobert Pettersson, Per Rosengren, Erik Rosengren, Stefan Holmgren, LarsSparf, Richard Berglind, Thomas Eriksson, Karl Erik Patrik Nordström,Gunnar Martin Fröjdh, Xiatao Wang and Remo Behdasht.

U.S. patent application Ser. No. 14/311,366 is a continuation of PCTPatent Application No. PCT/US14/40579, entitled OPTICAL TOUCH SCREENSand filed on Jun. 3, 2014 by inventors Robert Pettersson, Per Rosengren,Erik Rosengren, Stefan Holmgren, Lars Sparf, Richard Berglind, ThomasEriksson, Karl Erik Patrik Nordström, Gunnar Martin Fröjdh, Xiatao Wangand Remo Behdasht.

PCT Application No. PCT/US14/40579 claims priority benefit from:

-   -   U.S. Provisional Patent Application No. 61/950,868, entitled        OPTICAL TOUCH SCREENS and filed on Mar. 11, 2014 by inventors        Karl Erik Patrik Nordström, Per Rosengren, Stefan Holmgren, Erik        Rosengren, Robert Pettersson, Lars Sparf and Thomas Eriksson;    -   U.S. Provisional Patent Application No. 61/923,775, entitled        MULTI-TOUCH OPTICAL TOUCH SCREENS WITHOUT GHOST POINTS and filed        on Jan. 6, 2014 by inventors Per Rosengren, Stefan Holmgren,        Erik Rosengren, Robert Pettersson, Lars Sparf and Thomas        Eriksson;    -   U.S. Provisional Patent Application No. 61/919,759, entitled        OPTICAL TOUCH SCREENS WITH TOUCH-SENSITIVE BORDERS and filed on        Dec. 22, 2013 by inventors Remo Behdasht, Erik Rosengren, Robert        Pettersson, Lars Sparf and Thomas Eriksson;    -   U.S. Provisional Patent Application No. 61/911,915, entitled        CIRCULAR MULTI-TOUCH OPTICAL TOUCH SCREENS and filed on Dec. 4,        2013 by inventors Richard Berglind, Erik Rosengren, Robert        Pettersson, Lars Sparf, Thomas Eriksson, Gunnar Martin Fröjdh        and Xiatao Wang;    -   U.S. Provisional Patent Application No. 61/833,161, entitled        CIRCULAR MULTI-TOUCH OPTICAL TOUCH SCREENS and filed on Jun. 10,        2013 by inventors Richard Berglind, Erik Rosengren, Robert        Pettersson, Lars Sparf, Thomas Eriksson, Gunnar Martin Fröjdh        and Xiatao Wang; and    -   U.S. Provisional Patent Application No. 61/830,671, entitled        MULTI-TOUCH OPTICAL TOUCH SCREENS WITHOUT GHOST POINTS and filed        on Jun. 4, 2013 by inventors Erik Rosengren, Robert Pettersson,        Lars Sparf and Thomas Eriksson.

U.S. patent application Ser. No. 15/000,815 is a continuation-in-part ofU.S. patent application Ser. No. 14/880,231, entitled GAMING DEVICE andfiled on Oct. 11, 2015 by inventors Stefan Holmgren, Sairam Iyer,Richard Berglind, Karl Erik Patrik Nordström, Lars Sparf, Per Rosengren,Erik Rosengren, John Karlsson, Thomas Eriksson, Alexander Jubner, RemoBehdasht, Simon Fellin, Robin Åman and Joseph Shain.

U.S. patent application Ser. No. 14/880,231 is a divisional of U.S.patent application Ser. No. 14/312,787, now U.S. Pat. No. 9,164,625,entitled OPTICAL PROXIMITY SENSORS and filed on Jun. 24, 2014 byinventors Stefan Holmgren, Sairam Iyer, Richard Berglind, Karl ErikPatrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren, JohnKarlsson, Thomas Eriksson, Alexander Jubner, Remo Behdasht, SimonFellin, Robin Åman and Joseph Shain.

U.S. patent application Ser. No. 15/000,815 is a continuation-in-part ofU.S. patent application Ser. No. 14/555,731, now U.S. Pat. No.9,741,184, entitled DOOR HANDLE WITH OPTICAL PROXIMITY SENSORS and filedon Nov. 28, 2014 by inventors Sairam Iyer, Stefan Holmgren and PerRosengren.

U.S. patent application Ser. No. 15/000,815 is a continuation-in-part ofU.S. patent application Ser. No. 14/791,414, entitled OPTICAL PROXIMITYSENSOR FOR TOUCH SCREEN AND ASSOCIATED CALIBRATION TOOL and filed onJul. 4, 2015 by inventors Per Rosengren, Xiatao Wang and StefanHolmgren.

U.S. patent application Ser. No. 14/791,414 claims priority benefit ofU.S. Provisional Patent Application Ser. No. 62/021,125, entitledOPTICAL TOUCH SCREEN SYSTEMS and filed on Jul. 5, 2014 by inventor PerRosengren.

U.S. patent application Ser. No. 14/312,787 is a continuation-in-part ofU.S. patent application Ser. No. 13/775,269, now U.S. Pat. No.8,917,239, entitled REMOVABLE PROTECTIVE COVER WITH EMBEDDED PROXIMITYSENSORS and filed on Feb. 25, 2013 by inventors Thomas Eriksson, StefanHolmgren, John Karlsson, Remo Behdasht, Erik Rosengren and Lars Sparf.

U.S. patent application Ser. No. 14/312,787 is also a continuation ofPCT Application No. PCT/US14/40112, entitled OPTICAL PROXIMITY SENSORSand filed on May 30, 2014 by inventors Stefan Holmgren, Sairam Iyer,Richard Berglind, Karl Erik Patrik Nordström, Lars Sparf, Per Rosengren,Erik Rosengren, John Karlsson, Thomas Eriksson, Alexander Jubner, RemoBehdasht, Simon Fellin, Robin Åman and Joseph Shain.

PCT Application No. PCT/US14/40112 claims priority benefit from:

-   -   U.S. Provisional Patent Application No. 61/986,341, entitled        OPTICAL TOUCH SCREEN SYSTEMS and filed on Apr. 30, 2014 by        inventors Sairam Iyer, Karl Erik Patrik Nordström, Lars Sparf,        Per Rosengren, Erik Rosengren, Thomas Eriksson, Alexander Jubner        and Joseph Shain;    -   U.S. Provisional Patent Application No. 61/972,435, entitled        OPTICAL TOUCH SCREEN SYSTEMS and filed on Mar. 31, 2014 by        inventors Sairam Iyer, Karl Erik Patrik Nordström, Lars Sparf,        Per Rosengren, Erik Rosengren, Thomas Eriksson, Alexander Jubner        and Joseph Shain;    -   U.S. Provisional Patent Application No. 61/929,992, entitled        CLOUD GAMING USER INTERFACE and filed on Jan. 22, 2014 by        inventors Thomas Eriksson, Stefan Holmgren, John Karlsson, Remo        Behdasht, Erik Rosengren, Lars Sparf and Alexander Jubner;    -   U.S. Provisional Patent Application No. 61/846,089, entitled        PROXIMITY SENSOR FOR LAPTOP COMPUTER AND ASSOCIATED USER        INTERFACE and filed on Jul. 15, 2013 by inventors Richard        Berglind, Thomas Eriksson, Simon Fellin, Per Rosengren, Lars        Sparf, Erik Rosengren, Joseph Shain, Stefan Holmgren, John        Karlsson and Remo Behdasht;    -   U.S. Provisional Patent Application No. 61/838,296, entitled        OPTICAL GAME ACCESSORIES USING REFLECTED LIGHT and filed on Jun.        23, 2013 by inventors Per Rosengren, Lars Sparf, Erik Rosengren,        Thomas Eriksson, Joseph Shain, Stefan Holmgren, John Karlsson        and Remo Behdasht; and    -   U.S. Provisional Patent Application No. 61/828,713, entitled        OPTICAL TOUCH SCREEN SYSTEMS USING REFLECTED LIGHT and filed on        May 30, 2013 by inventors Per Rosengren, Lars Sparf, Erik        Rosengren and Thomas Eriksson.

The contents of these applications are hereby incorporated by referencein their entireties.

FIELD OF THE INVENTION

The field of the present invention is light-based touch screens andproximity sensors and applications therefor, including door lock systemsthat utilize gestures to lock and unlock the door, a proximity sensorbar that is magnetically attached to a laptop display for enabling touchinput on a display that does not detect touch gestures, user interfacesfor in-vehicle infotainment systems, and user interfaces for PCs.

BACKGROUND OF THE INVENTION

In the prior art, a one-dimensional array of proximity sensors is notaccurate enough to determine a two-dimensional location of a pointerwithin a two dimensional plane extending from the array.

In prior art door lock systems, a portable wireless transmitter held bya person sends a coded signal to a wireless receiver connected to a doorlock mechanism to lock or unlock the door. Some prior art transmitterunits include switches for activating the lock and unlock functions,whereas other transmitter units are in the form of an electronictransponder card, whereby a transmitter unit connected to the lockinterrogates the transponder when a wake up signal is detected.

In order to provide an added level of security, some systems require theuser to enter a predefined authentication gesture to confirm that anauthorized person is trying to unlock the door of a vehicle. Thus, forexample, when the user presses a switch on a key fob transmitter, thatuser must enter a predefined authentication gesture on a touch sensor inorder to unlock the door. In another example, a detected predefinedauthentication gesture activates a transmitter unit to interrogate ahands-free card transponder.

Laptop computers are typically available in touchscreen andnon-touchscreen versions. It would be advantageous to enable consumersof non-touchscreen laptops to enable touchscreen functionality whendesired. For example, it would be advantageous to enable swipe, pinchand rotate gestures when browsing images, checking a newsfeed orrotating images. Another example is to enable touchscreen functionalityduring travel in an airplane where it is more comfortable to use one'sfingers on the screen than using the laptop's built-in trackpad.

Many in-vehicle infotainment systems employ touch screen user interfacesdesigned for handheld devices, such as mobile phones. It would beadvantageous to provide a user interface that is designed for the usecase of a display that is not held in the user's hand. It would beadditionally advantageous to provide user interfaces for electronicdevices, including handheld devices, desktop devices and in-vehicledevices that provide different schemes concurrently for accessingfunctions.

SUMMARY

Robot measurements indicate that there is a pattern in the relativesignal strengths that repeat within triangles spanned by three adjacentsignals. The robot measurement is used to learn that pattern, so that amapping is made from the relative signal strengths of three signals in atriangle, to the reflection location and strength of an obstacle withinthat triangle. Adjacent triangles give individual detection candidates,which are consolidated into one.

There is thus provided in accordance with an embodiment of the presentinvention a proximity sensor for identifying a proximal object,including a housing, a plurality of light emitters mounted in thehousing for projecting light out of the housing, a plurality of lightdetectors mounted in the housing, operable when activated to detectamounts of light arriving at the detectors, a plurality of lensesmounted in the housing, each lens, denoted L, being positioned inrelation to two respective ones of the detectors, denoted D1 and D2,such that light entering lens L is maximally detected at detector D1when the light enters lens L at an acute angle of incidence θ1, andlight entering lens L is maximally detected at detector D2 when thelight enters lens L at an obtuse angle of incidence θ2, and a processorconnected to the emitters and to the detectors, operable tosynchronously activate emitter-detector pairs, and configured tocalculate a partial contour of an object outside the housing thatreflects light, projected by the activated emitters, back towards saidlenses, based on amounts of light detected by the activated detectors.

There is additionally provided in accordance with an embodiment of thepresent invention a method for sensing a proximal object, includingproviding a strip comprising a plurality of emitters E and detectors Dwherein each emitter is situated between different detectors,synchronously co-activating emitter-detector pairs (E, D), wherein theemitters and detectors are arranged such that for each emitter-detectorpair (E, D), when an object is located at a target position p(E, D)corresponding to the pair (E, D), then the light emitted by emitter E isscattered by the object and is maximally detected by detector D,determining a reflection value R(E, D) for each emitter-detector pair(E, D), based on an amount of reflected light detected by detector Dwhen the pair (E, D) is activated by the synchronously co-activating,and associating the reflection value R(E, D) with the target positionp(E, D) in the common plane corresponding to the pair (E, D), generatinga two-dimensional pixel image of reflection values R_(p) at pixelpositions p, corresponding to the derived reflection values R(E, D) andthe target positions p(E, D), and estimating a partial circumference ofthe object based on the pixel image.

There is further provided in accordance with an embodiment of thepresent invention a monitor, including a housing, a display mounted inthe housing, a plurality of light emitters mounted in the housing forprojecting light out of the housing along two orthogonal detectionplanes, a plurality of light detectors mounted in the housing fordetecting reflections of the light projected by the emitters, by areflective object in one of the detection planes, a plurality of lensesmounted and oriented in the housing relative to the emitters and thedetectors in such a manner that for each emitter-detector pair, when theobject is located at a target position corresponding to thatemitter-detector pair, light emitted by the emitter of that pair passesthrough one of the lenses and is reflected by the object back throughone of the lenses to the detector of that pair, and a processorconnected to the display, to the emitters and to the detectors, fordisplaying a graphical user interface (GUI) on the display, forinterpreting different directional movements of the object performedacross the two orthogonal detection planes as different input commandsto the GUI, for synchronously co-activating emitter-detector pairs, andfor calculating a directional movement of the object in the twoorthogonal detection planes by determining a series of emitter-detectorpairs among the co-activated emitter-detector pairs, for which thedetector detects a maximum amount of light over a time interval, andidentifying the target positions corresponding thereto, and calculatinga direction of movement based on the thus-identified target positions.

There is yet further provided in accordance with an embodiment of thepresent invention a monitor, including a housing, a display, a pluralityof light emitters mounted in the housing for projecting light out of thehousing along a detection plane parallel to the display, a plurality oflight detectors mounted in the housing for detecting reflections of thelight projected by the emitters, by a reflective object in the detectionplane, a plurality of lenses mounted and oriented in the housingrelative to the emitters and the detectors in such a manner that foreach emitter-detector pair, when the object is located at a targetposition corresponding to that emitter-detector pair, then light emittedby the emitter of that pair passes through one of the lenses and isreflected by the object back through one of the lenses to the detectorof that pair, and a processor connected to the display, to the emittersand to the detectors, for displaying a graphical user interface on thedisplay for adjusting parameters for the display, for synchronouslyco-activating emitter-detector pairs, and for calculating a position ofthe object in the detection plane by determining an emitter-detectorpair among the co-activated emitter-detector pairs, for which thedetector detects a maximum amount of light over a time interval, andidentifying the target position corresponding thereto, determiningadditional target positions corresponding to co-activatedemitter-detector pairs, which neighbor the thus-identified targetposition, and calculating a weighted average of the target position andthe additional target positions, wherein each target position's weightin the average corresponds to a degree of detection of the reflectedlight beam for the emitter-detector pair to which that target positioncorresponds.

There is moreover provided in accordance with an embodiment of thepresent invention a calibration tool for calibrating parameters of aproximity sensor strip including a plurality of emitters E and detectorsD, wherein the emitters and detectors are arranged such that theemitters project light out of the strip along a detection plane and thedetectors detect light entering the strip along the detection plane, andfor each emitter-detector pair (E, D), when an object is located at atarget position p(E, D) in the detection plane, corresponding to thepair (E, D), then the light emitted by emitter E is scattered by theobject and is expected to be maximally detected by detector D, thecalibration tool including a reflective object placed parallel to theproximity sensor strip in the detection plane, the reflective objectspanning the length of the proximity sensor, a mechanism forincrementally moving the reflective object towards or away from theproximity sensor along the detection plane, and a processor coupled withthe proximity sensor strip and with the mechanism operable to (i)activate a plurality of the emitter-detector pairs (E, D) at eachincremental move of the reflective object, (ii) measure detectionsdetected by detector D of each activated pair, and (iii) calibrate thetarget positions p(E, D) in the detection plane according to thedistances between the sensor strip and the reflective object at whichmaximum detections are measured.

There is additionally provided in accordance with an embodiment of thepresent invention a method for calibrating parameters of a proximitysensor strip including a plurality of emitters E and detectors D,wherein the emitters and detectors are arranged such that the emittersproject light out of the strip along a detection plane and the detectorsdetect light entering the strip along the detection plane, and for eachemitter-detector pair (E, D), when the object is located at a targetposition p(E, D) in the detection plane, corresponding to the pair (E,D), then the light emitted by emitter E is scattered by the object andis expected to be maximally detected by detector D, the method includingproviding a reflective object spanning the length of the proximitysensor parallel to the proximity sensor strip in the detection plane,incrementally moving the reflective object towards or away from theproximity sensor along the detection plane, at each incremental move ofthe object, activating a plurality of the emitter-detector pairs (E, D)to measure detections at detectors D, and calibrating the targetpositions p(E, D) in the detection plane according to the distancesbetween the sensor strip and the reflective object at which maximumdetections are measured.

There is further provided in accordance with an embodiment of thepresent invention a proximity sensor for identifying a location of aproximal object, including a housing, a plurality of light emitters,denoted E, mounted in the housing for projecting light out of thehousing along a detection plane, a plurality of light detectors, denotedD, mounted in the housing, operable when activated to detect amounts oflight entering the housing along the detection plane, whereby for eachemitter-detector pair (E, D), when an object is located at a targetposition p(E, D) in the detection plane, corresponding to the pair (E,D), then the light emitted by emitter E is scattered by the object andis expected to be maximally detected by detector D, and a processorconnected to the emitters and to the detectors, operable tosynchronously activate emitter-detector pairs, to read the detectedamounts of light from the detectors, and to calculate a location of theobject in the detection plane from the detected amounts of light, inaccordance with a detection-location relationship, denoted D→L, thatrelates detections from emitter-detector pairs to object locationsbetween neighboring target positions in the detection plane.

There is yet further provided in accordance with an embodiment of thepresent invention a door lock system that enters an activatable state,whereby the lock is activated in response to detecting a firstnon-predefined gesture, and the lock is subsequently activated to unlockin response to that same gesture being detected again.

There is moreover provided in accordance with an embodiment of thepresent invention a proximity sensor array in an elongated housing thatis attached by a user to an edge of a laptop computer screen to providetouchscreen detection to the laptop. In some embodiments a UniversalSerial Bus (USB) connector extends from the elongated housing and isinserted into a USB socket in the laptop, enabling the proximity sensorto communicate with the laptop using USB communications protocols andalso enabling the proximity sensor to receive electric current from thelaptop. In some embodiments, the proximity sensor communicates with thelaptop wirelessly; e.g., using a short range wireless connectivitystandard. In some embodiments the elongated housing includes one or moremagnetic fasteners for attaching the proximity sensor array along anedge, e.g., the bottom edge, of the laptop screen.

There is additionally provided in accordance with an embodiment of thepresent invention a single straight bar including a linear array ofinterlaced light emitters and photodiode detectors mounted on a printedcircuit board, wherein the bar is configured to be repeatedly attachedto and detached from an exterior housing of a laptop computer includinga processor, wherein the bar, when thus attached and coupledcommunicatively with the laptop processor, provides the processor withdetection signals that enable the processor to recognize a plurality ofdifferent gestures performed by an object in an airspace of a projectionplane coming out of one side of the bar, the detection signals beinggenerated by light emitted by the light emitters that is reflected bythe object back to the bar and detected by the photodiodes.

There is further provided in accordance with an embodiment of thepresent invention a single straight bar including a linear array ofinterlaced light emitters and photodiode detectors mounted on a printedcircuit board, wherein the bar is configured to be repeatedly attachedto and detached from an exterior housing of a laptop computer includinga processor, wherein the bar, when coupled communicatively with thelaptop processor and positioned over one side of a flat rectangularsurface of the laptop housing, provides the processor with detectionsignals that enable the processor to recognize a plurality of differentgestures performed by an object in an airspace in front of the surface,the detection signals being generated by light emitted by the lightemitters that is reflected by the object back to the bar and detected bythe photodiodes.

Embodiments of the present invention provide two-dimensional (2D) touchdetection using a one-dimensional array of alternating light emittersand detectors. The present invention also provides a three-dimensional(3D) touch or hover detector based on the same principles as the 2Ddetectors.

There is additionally provided in accordance with an embodiment of thepresent invention a GUI for an in-vehicle infotainment system, providingboth context-driven navigation of the GUI and hierarchical menu-drivennavigation thereof, within a single display simultaneously.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be more fully understood and appreciated fromthe following detailed description, taken in conjunction with thedrawings in which:

FIG. 1 is a simplified illustration of light emitted from light sourcesalong the solid lines, and reflected along the dashed lines to lightsensors, in accordance with an embodiment of the present invention;

FIG. 2 is an illustration of backward and forward hotspot signal values,in accordance with an embodiment of the present invention;

FIG. 3 is an illustration of the signal value relationship betweentop-middle and center hotspots, in accordance with an embodiment of thepresent invention;

FIG. 4 is an illustration of the signal value relationship betweenright-middle and center hotspots, in accordance with an embodiment ofthe present invention;

FIG. 5 is an illustration of the signal value relationship betweentop-right and center backward hotspots, and top-middle and right-middleforward hotspots, in accordance with an embodiment of the presentinvention;

FIG. 6 is an illustration showing that the relationship between twosignal values v0 and v1 (solid lines) is expressed as r=log(v1)-log(v0)(dashed line), in accordance with an embodiment of the presentinvention;

FIG. 7 is an illustration using triangles to mark areas in which allspanning hotspot signal values are relatively strong, in accordance withan embodiment of the present invention;

FIG. 8 is an illustration showing detection errors across a 100×64 mmtouch screen, in accordance with an embodiment of the present invention;

FIG. 9 is an illustration showing a 2D histogram of sample errorvectors, in accordance with an embodiment of the present invention;

FIGS. 10-13 are simplified illustrations of a proximity sensor, inaccordance with an embodiment of the present invention;

FIGS. 14 and 15 are simplified illustrations of calibration tools forthe proximity sensor of FIGS. 10-13, in accordance with an embodiment ofthe present invention;

FIGS. 16 and 17 are simplified illustrations showing how the calibrationtool of FIG. 15 identifies how the emitters and detectors of theproximity sensor have been mounted therein, in accordance with anembodiment of the present invention;

FIG. 18 is a simplified illustration of a proximity sensor detecting aproximal object, in accordance with an embodiment of the presentinvention;

FIG. 19 is a simplified illustration of a two dimensional image ofdetection values, in accordance with an embodiment of the presentinvention;

FIGS. 20 and 21 are simplified illustrations of a detected reflectionvalue for an emitter-receiver pair that is not associated with thatpair's hotspot location, in accordance with an embodiment of the presentinvention;

FIG. 22 is a simplified illustration of a detected partial circumferenceof an object in a two dimensional image of detection values, inaccordance with an embodiment of the present invention;

FIG. 23 is a simplified illustration of a method of estimating a partialcircumference of an object, in accordance with an embodiment of thepresent invention;

FIG. 24 is an illustration of a saddle roof or hyperbolic paraboloidcorresponding to the emitter light paths and reflected light paths of a3-D proximity sensor, in accordance with an embodiment of the presentinvention;

FIG. 25 is a simplified illustration of a circular arrangement of sixemitters and six receivers arranged alternatingly along a circular basethat provide 30 hotpot locations along a 3-D hyperboloid, in accordancewith an embodiment of the present invention;

FIG. 26 is a simplified illustration of a grid representing emitted andreflected light beams from a circular arrangement of 16 emitters and 16receivers arranged alternatingly along the circle, which provide 176hotpot locations along a 3-D hyperboloid, in accordance with anembodiment of the present invention;

FIG. 27 is a simplified flowchart of a method of locking and unlocking adoor, in accordance with an embodiment of the present invention;

FIG. 28 is a simplified illustration of a car that practices thelock-and-unlock method of FIG. 27, in accordance with an embodiment ofthe present invention;

FIG. 29 is a simplified illustration of the interior of the car of FIG.28, in accordance with an embodiment of the present invention;

FIGS. 30 and 31 are simplified illustrations of a proximity sensor barconfigured as a laptop accessory, in accordance with an embodiment ofthe present invention;

FIGS. 32-36 are simplified illustrations of the laptop accessory ofFIGS. 30 and 31 placed along an edge of a laptop display, in accordancewith an embodiment of the present invention;

FIG. 37 is a simplified illustration of the laptop accessory of FIGS. 30and 31 situated along an edge of a laptop display and rotated away fromthe display to provide a detection plane in the airspace between thedisplay and the keyboard, in accordance with an embodiment of thepresent invention;

FIG. 38 is a simplified illustration of the laptop accessory of FIGS. 30and 31 situated along an edge of a laptop display and rotated away fromthe display to provide a detection plane along the surface of the laptopkeyboard, in accordance with an embodiment of the present invention;

FIG. 39 is a simplified flow chart of a process for assembling aproximity sensor, in accordance with an embodiment of the presentinvention;

FIG. 40 is a simplified illustration of a proximity sensor beingassembled according to the process of FIG. 39;

FIGS. 41 and 42 are simplified illustrations of light beams of aproximity sensor detecting an object, in accordance with an embodimentof the present invention;

FIG. 43 is a simplified illustration of a side view of a proximitysensor and light beams projected therefrom, in accordance with anembodiment of the present invention;

FIG. 44 is a simplified illustration of a proximity sensor lens andassociated optical components viewed from above and light beamsprojected through that lens, in accordance with an embodiment of thepresent invention;

FIG. 45 is a simplified illustration of a side view of the lens andcomponents of FIG. 44 and light beams projected through that lens, inaccordance with an embodiment of the present invention;

FIG. 46 is a simplified illustration of an L-shaped optical proximitysensor situated at a corner of a screen, in accordance with anembodiment of the present invention;

FIG. 47 is a simplified flow diagram of a method of identifying agesture, in accordance with an embodiment of the present invention;

FIG. 48 is a simplified illustration of a user interface, in accordancewith an embodiment of the present invention;

FIG. 49 is a simplified illustration of an optical proximity sensorsituated along a short segment of a display edge for adjusting displayparameters, in accordance with an embodiment of the present invention;

FIGS. 50, 53, 54 and 57 are flow charts for an in-vehicle infotainmentsystem GUI, in accordance with an embodiment of the present invention;

FIGS. 51, 55, 56 and 58 are screenshots of the in-vehicle infotainmentsystem GUI of FIGS. 50, 53, 54 and 57, in accordance with an embodimentof the present invention; and

FIG. 52 is an illustration of a notification card used in the in-vehicleinfotainment system GUI of FIGS. 50, 51 and 53-58, in accordance with anembodiment of the present invention.

The following table catalogs the numbered elements and lists the figuresin which each numbered element appears. Similarly numbered elementsrepresent elements of the same type, but they need not be identicalelements.

Numbered Elements Element Description Figures 101-116 emitter 10-18, 20,21, 23, 25, 40-45 201-211 photo-detector 10-18, 20, 21, 23, 25, 41, 42,45 301-304 lens 10-15, 17, 18, 20, 21 310-316 lens 31, 36, 40 321-327lens 41-45 401, 404 emitted light beam 10-18, 20, 21, 25 402, 403, 405,406 reflected light beam 10-12, 14-18, 20, 21, 25 410 proximity sensor25 light beams 411-414 proximity sensor 41-45 light beams 501 proximitysensor bar 10-18, 20, 21, 23, 28 502 motion sensor 26 503 portabletransponder 26 504 transmitter unit 26 510 proximity sensor bar 30-38511 housing 31 512 PCB 40 519, 520 proximity sensor 46, 48, 49 601, 602arrow 14, 15 701 processor 10-18, 20, 21, 23 801, 802 object 10-13, 18,20, 21, 23 803 motor 14, 15 804, 805 shaft 14, 15 806, 807 reflectiveobject 14-17 810 car 28 811 car door 28, 29 812 driver-side window 28,29 813 glove compartment 29 815, 816 seat 29 820 keyless entry 29 systemdoor lock 830 laptop 30, 32 831 display screen 30-38, 46, 48, 49 832keyboard 30-38 833 USB connector 30-34, 36 900-902 card menu item 52, 55903 menu bar 55, 56, 58 904-908 menu bar item 55, 56, 58 910-916, 916″,919, 919′, hotspot location 2-5, 10-13, 16-18, 926, 926″, 929, 929′,936, 20-23 936″, 939, 939′, 940-942, 944, 945, 961-969 971, 972detection plane 46, 48, 49 973 touch sensitive 49 portion of screen 974non-touch sensitive 49 portion of screen 975-977 icon 48 978-980 listitem 56, 58 981, 982 touch sensitive controls 49 983 touch location 49984 scroll knob 49 985 notification card 51, 58 986-988 application card51, 55, 56, 58 989 partial object 22 circumference 990 detection image19, 22 991-995 ring of six hotspot 25 locations in a hyperboloid 996sub-menu bar 56, 58 997-999 sub-menu bar item 56, 58 1001-1011 flowchartstep 27, 40 1020-1087 user interface action 50, 53, 54, 57

DETAILED DESCRIPTION

Throughout this description, the terms “source” and “emitter” are usedto indicate the same light emitting elements, inter alia LEDs, VCSELsand lasers, and the terms “sensor” and “detector” are used to indicatethe same light detecting elements, inter alia photo diodes.

Reference is made to FIG. 1, which is a simplified illustration of lightemitted from light sources along the solid lines, and reflected alongthe dashed lines to light sensors, in accordance with an embodiment ofthe present invention. FIG. 1 shows how light is emitted straight outfrom sources in collimated beams. Light that hits an obstacle isreflected diffusely. Sensors detect incoming light from reflections intwo narrow corridors that reach out from the sensor in two fixeddirections—both at the same angle away from opposite sides of the lightbeams, but on opposite sides of each beam, e.g., 30° and −30° from theemitter beam.

The amount of light that travels from one source to a sensor depends onhow centered the obstacle is on the source's beam, and how centered itis on one of the sensor's corridors. Such a source/sensor pair isreferred to as a “hotspot”. The obstacle location that gives the highestamount of light for a hotspot is referred to as the “hotspot location”or the “target position” for that source/sensor pair. A proximity sensoraccording to the present invention measures the transmitted amount oflight for each hotspot, and each such measurement is referred to as a“hotspot signal value”. The measurement normalizes all hotspot signalvalues so as to have the same range.

Since light that hits an obstacle is reflected diffusely and reflectionsare maximally detected in two narrow corridors at opposite sides of thelight beams, the present specification refers to a forward directiondetection based on all of the narrow detection corridors in a firstdirection, and a backward direction detection based on all of the narrowdetection corridors in the second direction. Stated differently, theforward direction includes all detections of emitter-detector pairs inwhich the detector of the pair has a higher location index than theemitter of the pair, and the backward direction includes all detectionsof emitter-detector pairs in which the detector of the pair has a lowerlocation index than the emitter of the pair. The forward direction maybe left or right, depending on device orientation. A hotspot where thesensor looks in the backward direction is referred to as a “backwardhotspot”, and vice versa for those looking forward.

Reference is made to FIG. 2, which is an illustration of backward andforward hotspot signal values, i.e., signal values for emitter-detectorpairs, in accordance with an embodiment of the present invention.Hotspot signal values are sampled with an obstacle placed at locationsin a dense grid spanning all hotspot locations; i.e., all locations atwhich an object can be placed such that the emitter-detector pairs willdetect a reflection value. FIG. 2 shows the maximum of all hotspotsignal values at obstacle locations within a region that spans 3×3hotspot locations, or target positions, separately for backward andforward hotspots. In FIGS. 2-5 hotspot locations are indicated asnumbered elements 961-969 only in the illustrations of backward hotspotsignal values. In FIGS. 2-5 the hotspot locations in the illustrationsof forward hotspot signal values are not indicated as numbered elementsin order to avoid cluttering the figure.

Reference is made to FIG. 3, which is an illustration of the signalvalue relationship between top-middle and center hotspots within a 3×3grid of forward hotspots, i.e., hotspots at locations (2,1) and (2,2) inthe 3×3 grid, in accordance with an embodiment of the present invention.Reference is also made to FIG. 4, which is an illustration of the signalvalue relationship between right-middle and center hotspots within a 3×3grid of forward hotspots, i.e., hotspots at locations (3,2) and (2,2) inthe 3×3 grid, in accordance with an embodiment of the present invention.Reference is also made to FIG. 5, which is an illustration of the signalvalue relationship between top-right and center backward hotspots, andtop-middle and right-middle forward hotspots, each within a 3×3 grid ofhotspots, in accordance with an embodiment of the present invention.FIGS. 3-5 show relationships between two adjacent hotspot signal values.Each curve follows a fixed relationship value, similar to a topologicalmap. Reference is also made to FIG. 6, which is an illustration showingthat the relationship between two signal values v0 and v1 (solid lines)is expressed as a difference of logarithms of the values (dashed line),in accordance with an embodiment of the present invention. FIG. 6 showsthe relationship expressed as r=log(v1)-log(v0). This relationship isdrowned in noise when either of the signal values has a lowsignal-to-noise (SNR) ratio.

The signal value relationship between two vertically adjacent hotspotscorresponds to a curve in FIG. 3. If the signal values are assumed to benormally distributed with a certain standard deviation, then thatassumption may be used to find an interpolated location between thehotspot locations according to FIG. 6, referred to as a “crossing”. Itdoes the same for two vertically adjacent hotspots next to and at eitherside of the first, to create a second crossing. The rationale is thatthe obstacle location is somewhere between the two crossings. If thecurves in FIG. 3 are all straight and parallel, this would be accurate.However, curvature causes inaccuracy.

To account for such curvature, the location between the crossing isfound using the same method, but from the relationships of horizontallyadjacent hotspots. The curves are now those in FIG. 4. Instead offinding horizontal crossings and selecting the location between bothpairs of crossings, a shortcut is used. The vertical crossings arethought of as virtual hotspots, and each signal value is estimated basedon the real hotspot signal values and the relative distance to each. Thesignal value relationship of the crossing's virtual hotspots gives theobstacle location directly.

Since the hotspot signal values for all obstacle locations have beenrecorded by a robot, finding a new obstacle location is achieved byfinding the sample whose signals match those caused by the obstacle.This may not be efficient, though, due to high memory and high timecomplexity. Comparing the relationship between the highest signal valuesand those of adjacent hotspots should be sufficient.

Reference is made to FIG. 7, which is an illustration using triangles tomark areas in which all spanning hotspot signal values are relativelystrong, in accordance with an embodiment of the present invention. Themapping from two-dimensional signal relationships to three-dimensionallocation and reflectivity is similar in all triangles; especially so intriangles of the same orientation in the same horizontal band. Thismeans that the mapping needs to be learned and stored for only a fewtriangle groups. It may be observed in FIG. 2 that there are triangularareas spanned by three hotspots, in which those three hotspot signalvalues are all relatively high. Some of these are drawn in FIG. 7. Thismeans that the three pairwise relationships between those signals willbe above noise within the area. Out of those three relationships one isredundant, since it is derivable from the other two. Within such atriangle, two signal relationships map to a location within thattriangle. It also maps to the reflectivity of the obstacle relative tothe observed hotspot signal values. These triangular areas cover thewhole screen, so the location and reflectivity of an obstacle is foundby finding the triangle that is spanned by the hotspots with the highestsignal values, and mapping the signal relationships to location andreflectivity.

The mapping transform takes the vertical (FIG. 3) and diagonal (FIG. 5)signal relationships as input. The input 2D space, from minimum tomaximum observed in each dimension, is covered by a 9×9 grid of nodes.Each node contains a location expressed in a frame of reference spannedby the triangle's edges. The location may be slightly outside of thetriangle. It also contains a compensation factor that, when multipliedwith the highest signal value, gives the reflectivity of the obstacle.The four nodes closest to the input are interpolated with bi-linearinterpolation.

All hotspots that have a signal value above a certain threshold, andthat are stronger than all its eight neighbors, are evaluated forpossible detections. All six triangles that use the maximum hotspot arescreened as possible contributors to the detection. Each triangle isgiven a weight that is calculated as the product of all its hotspotsignal values. The highest three are kept, and their weights are reducedby that of the fourth highest. The kept triangles are evaluated, andtheir results are consolidated to a weighted average, using the weightsused for screening.

Finding strong signals around which to evaluate triangles, and tracking,may be performed as described in U.S. Pat. No. 9,164,625, entitledOPTICAL PROXIMITY SENSORS and filed on Jun. 24, 2014.

Using a robot to place a stylus at known locations opposite the sensorand recording the resulting detection signals, enables quantifyingaccuracy of the algorithm. The recorded sample signal values are sent asinput to the algorithm in random order, and the calculated detectionlocations based on these inputs are compared to the actual samplelocations.

Reference is made to FIG. 8, which is an illustration showing detectionerror across a 100×64 mm touch screen, in accordance with an embodimentof the present invention. The 2D error vector is color coded accordingto the legend at the right in FIG. 8. The legend circle radius is 5 mm.FIG. 8 shows how large, and in what direction, the error is for samplesacross the screen.

Reference is made to FIG. 9, which is an illustration showing a 2Dhistogram of sample error vectors, in accordance with an embodiment ofthe present invention. The units of the axes are mm. FIG. 9 shows thedistribution of the errors. TABLE I below provides the quantifiedaccuracy values.

Measurement Value Error distances 50:th percentile 0.43 mm Errordistances 95:th percentile 1.04 mm Error distances 99:th percentile 1.47mm True positives 100.0% False positives  0.0%

Reference is made to FIGS. 10-13, which are simplified illustrations ofa proximity sensor, in accordance with an embodiment of the presentinvention. FIGS. 10-13 show a proximity sensor 501, according to theteachings of the present invention. Proximity sensor 501 includes lightsources 101-110 and light sensors 201-211, each light source beingsituated between two of the sensors. Proximity sensor 501 also includesa plurality of lenses, such as lenses 301-304, each lens beingpositioned in relation to two respective neighboring ones of the sensorssuch that light entering that lens is maximally detected at a first ofthe two sensors when the light enters that lens at an acute angle ofincidence θ1, and light entering that lens is maximally detected at theother of the two sensors when the light enters that lens at an obtuseangle of incidence θ2. The lens is positioned in relation to the lightsource situated between these two sensors such that the light from thelight source is collimated as it exits proximity sensor 501. Thisarrangement provides the two narrow corridors that extend from eachsensor in two fixed directions away from opposite sides of the projectedlight beams discussed above with respect to FIG. 1.

FIG. 10 shows a forward reflection path of maximum detection for hotspot913 generated by source/sensor pair 104/207, whereby light from source104 reflected off object 801 is maximally detected by sensor 207, andFIG. 11 shows a backward reflection path of maximum detection forhotspot 914 generated by source/sensor pair 109/207, whereby light fromsource 109 reflected off object 801 is maximally detected by sensor 207.FIGS. 10 and 11 show how sensor 207 is situated with respect toneighboring lenses 303 and 304 such that sensor 207 receives maximumforward reflection values via lens 303, and maximum backward reflectionvalues via lens 304.

As explained above with respect to FIG. 1, the intersections outsideproximity sensor 501 between the projected light beams and the corridorsof maximum detection provide a map of hotspots. Four hotspots areillustrated in FIGS. 12 and 13, two of which are numbed 940 and 941. Anobject 801 is shown nearest hotspot 940 in FIG. 12. Thus, the maximumdetection of object 801 is generated by source/sensor pairs 104/202 and104/207. Source/sensor pair 104/202 provides backward detection, andsource/sensor pair 104/207 provides forward detection, as discussedabove. Additional detections are generated by other source/sensor pairs,e.g., forward detection source/sensor pair 105/208, because light beamsfrom source 105 are scattered, and a portion thereof arrives at sensor208, but the amount of light detected at sensor 208 is significantlyless than that generated by source/sensor pair 104/207, because thescattered light arriving at sensor 208 does not travel on the corridorof maximum detection.

FIG. 13 shows proximity sensor 501 of FIG. 12, but object 801 is moved adistance d to the right. In this case similar amounts of detection willbe generated by forward source/sensor pairs 104/207 and 105/208. Each ofthese detections will be less than the detection generated bysource/sensor pair 104/207 in FIG. 12 and greater than the detectiongenerated by source/sensor pair 105/208 in FIG. 12, as explained abovewith reference to FIGS. 3-7. The location of object 801 between hotspots 940 and 941 is calculated by interpolating the amounts of lightdetected by source/sensor pairs 104/207 and 105/208. As discussed abovewith reference to FIG. 7, a location of object 801 is calculated byperforming at least two interpolations between amounts of light detectedby source/sensor pairs that correspond to three neighboring hotspots,the neighboring hotspots being the vertices of a triangle in thedetection space.

In order to determine how to interpolate the detected amounts of light,detection sensitivities are calculated in the vicinities of the hotspotsusing a calibration tool that places a calibrating object having knownreflective properties at known locations in the detection zone outsideproximity sensor 501. At each known location, a plurality ofsource/sensor pairs is synchronously activated and amounts of lightdetected by neighboring activated sensors are measured. Repetitivepatterns in relative amounts of light detected by the neighboringactivated sensors as the object moves among the known location areidentified. These patterns are used to formulate detection sensitivitiesof proximity sensor 501 in the vicinities of the hotspots which are usedto determine how to interpolate the amounts of light detected in orderto calculate the location of a proximal object.

Reference is made to FIGS. 14 and 15, which are simplified illustrationsof calibration tools for the proximity sensor of FIGS. 10-13, inaccordance with an embodiment of the present invention. FIG. 14 shows afirst calibration tool that includes motor 803, and shafts 804 and 805that move reflective calibration object 806 horizontally and verticallyin relation to proximity sensor bar 501, as indicated by arrows 601 and602. At each location at which object 806 is placed, a plurality ofsource/sensor pairs that correspond to hotspots in the vicinity of thatlocation are activated and the amounts of light detected are used todetermine the sensitivity in the vicinity of those hotspots. Multiplesuch source/sensor pairs that share a common light source are activatedsimultaneously.

In some embodiments, the calibration tool, either that illustrated inFIG. 14 or that illustrated in FIG. 15, is used on certainrepresentative units of proximity sensor 501, and the interpolationmethods derived therefrom are applied to other similar units. In otherembodiments however, either calibration tool is used on each unit ofproximity sensor 501, in order to provide interpolations tailored toeach individual proximity sensor.

FIG. 15 shows a second calibration tool that differs from that of FIG.14 in the size and shape of the reflective calibration object. In FIG.14 calibration object 806 is modeled as a finger or stylus typicallyused with proximity sensor bar 501, whereas in FIG. 15 calibrationobject 807 is a rod that spans the length of proximity sensor bar 501.The rod is covered in a material having reflective properties similar tothose of skin or of a stylus typically used with proximity sensor bar501. In the calibration tool of FIG. 15, shaft 805 remains at a fixedlocation on shaft 804, such that object 807 only moves toward and awayfrom proximity sensor bar 501, as indicated by arrows 602. In this case,at each location of object 807 the light sources are activated one afterthe other and, during each light source activation, any of the lightsensors 201-211 that may reasonably be expected to detect a significantreflection therefrom are activated. In some embodiments, all of thelight sensors 201-211 are simultaneously activated with each lightsource activation.

In addition to determining interpolation methods, the calibration toolsare used to map the locations of the hotspots that correspond to thesource/sensor pairs. Often the locations of the hotspots are shiftedfrom their expected locations due to mechanical issues such as impreciseplacement or alignment of a light source or light detector withinproximity sensor 501. When used to this end, numerous proximity sensorunits need to be calibrated and the calibration tool of FIG. 15 is moreefficient than that of FIG. 14.

Reference is made to FIGS. 16 and 17, which are simplified illustrationsshowing how the calibration tool of FIG. 15 identifies how the emittersand detectors of the proximity sensor have been mounted therein, inaccordance with an embodiment of the present invention. FIGS. 16 and 17show how imprecise placement of a light sensor (FIG. 16) or a lightsource (FIG. 17) is identified by the calibration tool of FIG. 15. FIG.16 shows three rows of hot spots including hotspots 910-912, 919, 929,and 939. These are expected hot spot locations, i.e., proximity sensor501 is designed to provide maximum detections of reflected light forrespective activated source/sensor pairs when an object is placed atthese locations. This is verified as calibration rod 807 moves closer toproximity sensor 501. Each row of hot spots is situated at a fixeddistance from proximity sensor 501. Three distances are shown: H1, H2and H3.

FIG. 16 shows how, when light sensor 207 is placed slightly to the leftof its correct position within proximity sensor 501, maximum detectionmeasured at this light sensor corresponds to hotspot locations 919′,929′ and 939′. Calibration rod 807 enters these positions at differentdistances than those expected. FIG. 16 shows how calibration rod 807arrives at hotspot location 919′ when it is a distance H3′ fromproximity sensor 501. By analyzing a series of local maximum detectionsthat share a common light sensor and occur at different distances thanthose expected, the calibration system detects the offset of a lightsensor from its expected position. In some embodiments processor 701controls, or receives input from, motor 803 and processor 701 updatesits map of hotspots according to the actual local maximum detections.

FIG. 17 shows how, when light source 104 is placed slightly to the leftof its correct position within proximity sensor 501, maximum detectionmeasured for source/sensor pairs that include light source 104 areshifted from expected hotspot locations 916, 926 and 936, to positions916′, 926′ and 936′. FIG. 17 shows how calibration rod 807 arrives athot spot position 916′ when it is a distance H3′ from proximity sensor501. By analyzing a series of local maximum detections that share acommon light source and occur at different distances than thoseexpected, the calibration system detects the offset of the light sourcefrom its expected position.

A proximity sensor according to the present invention is used toestimate a partial circumference of a proximal object. Reference is madeto FIG. 18, which is a simplified illustration of a proximity sensordetecting a proximal object, in accordance with an embodiment of thepresent invention. FIG. 18 shows proximity sensor strip 501 and proximalobject 802. Four hotspot locations 939-942 along the edge of object 802facing the sensor are shown. The reflection values associated with thesehotspot locations are used to estimate the contour of this edge.

As described above, each hotspot location is associated with one or twosource/sensor pairs. In FIG. 18, hotspot location 940 is associated withsource/sensor pairs 104/202 and 104/207.

The reflection values are used to generate a two-dimensional pixel imageof reflection values indicating where reflective surfaces arepositioned. For example, when all hotspot locations for allsource/sensor pairs in proximity sensor 501 are assigned theirrespective, normalized reflection values, the result is atwo-dimensional image. The reflection values in different embodimentsare normalized within a range determined by the number of bits providedfor each pixel in the two-dimensional image, e.g., 0-255 for 8-bit pixelvalues, and 0-1023 for 10-bit pixel values.

Reference is made to FIG. 19, which is a simplified illustration of atwo dimensional image of detection values, in accordance with anembodiment of the present invention. FIG. 19 shows proximity sensor 501whose detection plane is directed downward and the resultingtwo-dimensional image 990 of reflection values generated by an objectsituated within that detection plane. The pixel values in image 990 are8-bit values.

Because more than one source/sensor pair corresponds to each hotspotlocation, the reflection value for that location in the two-dimensionalimage can be derived in different ways. Namely, the forward-directionsource/sensor pair can be used, or the backward-direction source/sensorpair can be used. In some embodiments, the average of these two valuesis used, and in other embodiments the maximum of these two values isused, such that some pixels derive their values from forward-directionsource/sensor pairs, and other pixels derive their values frombackward-direction source/sensor pairs.

Certain reflection values for source/sensor pairs are not caused by areflective object at the corresponding hotspot, but rather by strayreflections at entirely different locations. FIGS. 20 and 21 show howthese cases are identified. Once identified, the corresponding pixelvalues in the two-dimensional image are reset to zero.

Reference is made to FIGS. 20 and 21, which are simplified illustrationsof a detected reflection value for an emitter-receiver pair that is notassociated with that pair's hotspot location, in accordance with anembodiment of the present invention. FIG. 20 shows hotspot locations 940and 944 aligned along a common emitter beam path 401. Hotspot location940 corresponds to source/sensor pairs 104/202 and 104/207, and hotspotlocation 944 corresponds to source/sensor pairs 104/201 and 104/208. Itis clear from FIG. 20 that any light from emitter 104 is blocked byobject 802 well before it arrives at hotspot location 944, and thereforeany light detected at sensors 201 and 208 during activation of source104 is not generated by a reflective object at hotspot location 944, butis rather stray reflections off the object at other locations.Therefore, the reflection value appropriate for hotspot location 944 iszero.

This state is determined by the fact that source/sensor pair 104/202 hasa significant detected reflection value, indicating that a reflectiveobject is at corresponding location 940, and therefore, light beam 401does not arrive at location 944. Moreover, because the lenses and thesensors are configured such that the maximum detection arrives at thesensor when it is reflected at angle θ1 it is clear that thesource/sensor pair detecting the maximum reflection from among allsource/sensor pairs that share a common source is the pair detectingreflections from an object at, or near, the corresponding hotspotlocation. Indeed, in the example shown in FIG. 20 the detection valuefor source/sensor pair 104/202 is much greater than the detection valuefor source/sensor pair 104/201. For the same reason, the detection valuefor source/sensor pair 104/207 is much greater than the detection valuefor source/sensor pair 104/208. A similar situation is illustrated inFIG. 21, except that in this case the two hotspot locations are situatedalong a common detection path.

FIG. 21 shows hotspot locations 940 and 945 aligned along a commonmaximum detected reflection path 403. Hotspot location 940 correspondsto source/sensor pair 104/202, and hotspot location 945 corresponds tosource/sensor pair 105/202. It is clear from FIG. 21 that light fromonly one location can be reflected along path 403 onto receiver 202. Andbecause the detected reflection value for source/sensor pair 104/202 isgreater than the detection value for source/sensor pair 105/202, it issafe to assume that the reflecting object is at, or near, hotspotlocation 940, and the detection value for source/sensor pair 105/202 isnot caused by a reflective object at hotspot location 945. Therefore,the reflection value appropriate for hotspot location 945 is zero.

In general, an emitted light path LP, such as path 401 in FIG. 17, has aplurality of hotspot locations thereon, denoted P₁, P₂, . . . , P_(N),at different distances from the proximity sensor 501, such as hotspotlocations 916, 926 and 936, in FIG. 17. When an object is located at oneof these locations, denoted P_(i) other hotspot locations P_(i+j) andP_(i−k) also have corresponding detection values. In such cases, thehotspot location P_(max) for which a maximum detection value is detectedfrom among hotspot locations along LP, is considered to correspond tothe object, and all detection values for hotpot locations further fromproximity sensor 501 are reset to zero. Detection values for hotpotlocations between P_(max) and proximity sensor 501 are retained. Often,two hotspot locations P_(max) and P_(max+1) are used to calculate thelocation of the object, as explained hereinabove, and in such casesP_(max+1) is not reset to zero.

Similarly, a reflected light path RP, such as path 402 in FIG. 16, has aplurality of hotspot locations thereon, denoted P₁, P₂, . . . P_(N), atdifferent distances from the proximity sensor 501, such as hotspotlocations 919, 929 and 939, in FIG. 16. When an object is located at oneof these locations, denoted P_(i) other hotspot locations P_(i+j) andP_(i−k) also have corresponding detection values. In such cases, thehotspot location P_(max) for which a maximum detection value is detectedfrom among hotspot locations along RP, is considered to correspond tothe object, and all detection values for hotpot locations further fromproximity sensor 501 are reset to zero. Detection values for hotpotlocations between P_(max) and proximity sensor 501 are retained. Often,two hotspot location P_(max) and P_(max+1) are used to calculate thelocation of the object, as explained hereinabove, and in such casesP_(max+1) is not reset to zero.

In this manner, the two-dimensional pixel image is refined and begins torepresent the contour of the object facing the sensor. Reference is madeto FIG. 22, which is a simplified illustration of a detected partialcircumference of an object in a two dimensional image of detectionvalues, in accordance with an embodiment of the present invention. FIG.22 shows the detected partial circumference 989 in the detection image990 of FIG. 19 and an example pixel 915 having a non-zero detectionvalue, but whose appropriate reflection value is zero, as explainedhereinabove.

The next step is to filter the pixels in this image to obtain sub-pixelprecision for the location of the object's contour between hotspotlocations. After calculating sub-pixel values, various edge detectionfilters are applied to the two-dimensional pixel image to identify theedges of the object facing the sensor and discard stray reflections.Known edge detection filters include Sobel, Canny, Prewitt, Laplace,gradient. This edge information is used to determine a length of thisportion of the object, i.e., a partial circumference of the object, andits location.

The length of the detected portion of the object is calculated usingdifferent methods, in accordance with different embodiments of theinvention. Some embodiments determine the number of pixels, orsub-pixels, along the detected portion of the object. Other embodimentscalculate the sum of the distances between each pair of neighboringpixels, or sub-pixels, along the detected portion of the object. Stillother embodiments determine an equation for a curve that passes througheach of the pixels, or sub-pixels, along the detected portion of theobject, and calculates the length of the partial circumference of theobject according to this equation.

In some embodiments, in order to relax processor complexity, an estimateof the partial circumference is calculated based on three points: thepoint on the object for which there is a maximum detection value and thetwo outermost points along the partial circumference.

Reference is made to FIG. 23, which is a simplified illustration of amethod of estimating a partial circumference of an object, in accordancewith an embodiment of the present invention. FIG. 23 shows point 940 forwhich there is a maximum detection value and two outermost points 939and 941 along the partial circumference of object 802. An estimate ofthe partial circumference is the sum of the distances from point 939 topoint 940 and from point 941 to point 940. In order to further refinethis calculation, without adding much complexity to the calculations,the system calculates the sub-pixel coordinates of these three positionsusing the immediate neighbors of the respective hotspot locations939-941, but does not calculate sub-pixel locations for any other pixelsin the two-dimensional pixel image. Point 940, or a respective sub-pixellocation, for which there is a maximum detection value is used as theobject's coordinates.

In other embodiments of the invention, the shape of the proximity sensoris not a straight line, but circular, or wave-shaped to provide a 3-Ddetection volume, instead of a 2-D detection plane. In such alternativeembodiments, the emitters and receivers are still alternated as they arein proximity sensor 501, and each emitter is paired with each of thereceivers as a source/sensor pair having a corresponding hotspot withina 3D volume above the proximity sensor.

Reference is made to FIG. 24, which is an illustration of a saddle roofor hyperbolic paraboloid corresponding to the emitter light paths andreflected light paths of a 3-D proximity sensor, in accordance with anembodiment of the present invention.

Reference is made to FIG. 25, which is a simplified illustration of acircular arrangement of six emitters and six receivers arrangedalternatingly along a circular base that provides 30 hotpot locationsalong a 3-D hyperboloid, in accordance with an embodiment of the presentinvention. FIG. 25 shows emitters 101 and 102, and receivers 201 and202, which provide 30 hotpot locations along a 3-D hyperboloid, inaccordance with an embodiment of the present invention. FIG. 25 showsfive rings, 991-995, of hotspot locations along the height of thehyperboloid.

Reference is made to FIG. 26, which is a simplified illustration of agrid representing emitted and reflected light beams from a circulararrangement of 16 emitters and 16 receivers arranged alternatingly alonga circular base, which provide 176 hotpot locations along a 3-Dhyperboloid, in accordance with an embodiment of the present invention.Alternative configurations include 4 emitters and 4 receivers for astripped down hyperboloid, 3 emitters and 3 receivers for a regularoctahedron and 2 emitters and 2 receivers for a regular tetrahedron.These 3-D proximity sensors are used inter alia for detecting in-airhand wave gestures.

Proximity sensors according to the present invention have numerousapplications for touch screens, control panels and new user interfacesurfaces. The proximity sensor can be mounted anywhere—on a wall, awindow, placed on a notebook, and it will provide touch and gesturedetection upon that item. These detected gestures are then used as inputto electronic systems. For example, a gesture along a wall can dim thelighting in the room by mounting the sensor along an edge of the walland communicating the detected gestures to the lighting system.Significantly, the proximity sensor is only mounted along one edge ofthe detection area, reducing component cost and providing moreflexibility for industrial design of touch screens and touch sensitivecontrol panels.

A door lock system according to the present invention has two modes ofoperating. In the first mode, the door is locked and unlocked usingprior art methods, such as by a transponder signal, by pressing a keyfob switch, or by a physical key inserted into a keyhole and rotated. Inthe second mode, the user locks the door by entering a gesture on agesture sensor. The user subsequently unlocks the door by entering thatsame gesture on the gesture sensor. However, unlike prior artgesture-based lock systems, this unlock gesture is not defined as anunlock gesture until the user enters it to lock the door.

Reference is made to FIG. 27, which is a simplified flowchart of amethod of locking and unlocking a door, such as a car door, inaccordance with an embodiment of the present invention. The methodbegins at step 1001 when the door is closed. At step 1002 the closing ofthe door is detected by the door lock system and this triggers thesystem to activate a transmitter unit connected to the lock system tolocate a portable transponder, and specifically, to identify whetherthat transponder is behind the closed door, inside the closed interior,e.g., inside the car. If that transponder is inside the closed interior,the system activates a gesture detection apparatus directed at detectinggestures on, or near, the exterior of the door. At step 1003 if thegesture detection apparatus detects a gesture, then at step 1004 thedoor lock is activated and the detected gesture is stored to memory. Atstep 1005 the stored gesture is detected again by the gesture detectionapparatus, and at step 1006 the door is unlocked.

In certain embodiments where multiple doors provide access to a commoninterior space, e.g., a car, the lock system only proceeds from step1001 if at step 1001 a all other doors to the common interior are closedwhen the closing of the door is detected.

In some embodiments, the gesture detection apparatus is activated whenthe door is closed only if at step 1001 b it is further determined thatno one remains inside the closed space. This is determined usingmovement sensors, cameras or other means known to these skilled in theart.

In some embodiments, the gesture detection apparatus is activated whenthe door is closed, without identifying whether a transponder is insidethe closed interior. This enables using the gesture lock-unlock methodaccording to the present invention in systems that do not includetransponders, and also enables using the gesture lock-unlock method whenthe user removes the transponder from the closed space before closingthe door.

In some embodiments the gesture detection apparatus is the proximitysensor strip described hereinabove mounted along an edge of thedriver-side window to detect gestures made on the exterior of thewindow. In other embodiments, other types of gesture detection apparatusare provided, inter alia, cameras, optical touch screens based onblocked light, optical touch screens based on frustrated total internalreflection (FTIR), optical proximity sensors, capacitive touch screensand resistive touch screens. In other embodiments, the gesture detectionapparatus detects gestures on a wall next to the door, on the doorhandle or doorknob, or in the open space in front of the door.

Different systems enable detecting different types of gestures. Examplegestures include: touching one or more locations on a surface; one ormore two-dimensional lines or squiggles traced by a finger on a surface;one or more two-dimensional lines or squiggles traced by multiplefingers on a surface, e.g., multi-finger pinch, spread or rotationgestures; hand wave gestures; holding up a number of fingers; signlanguage gestures; and full body movements.

Reference is made to FIG. 28, which is a simplified illustration of acar that practices the lock-and-unlock method of FIG. 27, in accordancewith an embodiment of the present invention. FIG. 28 shows a car 810having a door 811 and a side window 812. A proximity sensor strip 501 ismounted beneath window 812 and projects light beams 410 along the outersurface of window 812 to detect gestures on the window, e.g., a shapetraced by a finger gliding upon the window.

In some embodiments, the stored gesture includes the shape traced by thefinger, but not the location on window 812 at which the shape wasoriginally traced. In other embodiments, the location on window 812 atwhich the shape was originally traced is also stored, and the user mustrecreate the gesture at the same location in order to unlock the door.

Reference is made to FIG. 29, which is a simplified illustration of theinterior of car 810, in accordance with an embodiment of the presentinvention. FIG. 29 shows driver seat 815, passenger seat 816, glovecompartment 813, driver-side door 811 and driver-side window 812. Aportable electronic transponder 503 is shown inside glove compartment813. A transmitter unit 504 is shown mounted in door 811 to interrogatetransponder 503 when the door is closed and determine whether thetransponder is inside the car or outside. If the transponder is insidethe car, transmitter unit 504 communicates this to keyless entry system820 which then activates the gesture detection apparatus (not shown inFIG. 29).

FIG. 29 also shows motion sensor 502 mounted in the car dashboard fordetecting if any people remain in the car after door 811 is closed.Sensor 502 is also in communication with keyless entry system 820.Keyless entry system 820 will only enable the gesture detectionapparatus to activate the lock if no one is inside the car.

In accordance with an embodiment of the present invention, a laptopaccessory is provided that enables converting a non-touchscreen laptopinto a touchscreen laptop. The accessory is a proximity sensor barfeaturing an elongated proximity sensor array. Although this proximitysensor bar is described as an accessory for laptop computers, it isuseful for other computer displays, inter alia, all-in-one computers,desktop computers, tablets and televisions. It is also useful forconverting any surface, including non-display surfaces, such as a table,wall, or window, into a touch sensitive surface on which gestures areperformed to control an electronic device. The proximity sensor barincludes any of the proximity sensors discussed hereinabove, inter alia,with reference to FIGS. 10-26, incorporated into a touch sensoraccessory that communicates user interface commands to a separateelectronic device.

Reference is made to FIGS. 30 and 31, which are simplified illustrationsof a proximity sensor bar configured as a laptop accessory, inaccordance with an embodiment of the present invention. FIGS. 30 and 31show proximity sensor bar 510 connected to laptop computer 830 via wire833. Typically a USB connector situated at the end of wire 833 isinserted into a USB socket in laptop 830. Laptop computer 830 includesdisplay 831 and keyboard 832. The operating system running on laptop 830supports touchscreen user interface commands enabling communicationbetween proximity sensor bar 510 connected to laptop computer 830 usinga USB-HID digitizer. This enables the proximity sensor bar to mapmultiple touch coordinates to the screen, and send those coordinates tothe laptop which interprets those coordinates into one or more gestures.In some embodiments, proximity sensor bar 510 is configured to providemouse or trackpad functionality by reporting detected object coordinatesas a single finger trackpad. In certain embodiments, proximity sensorbar 510 is configured to interpret gestures into commands, such asrotate, scroll, zoom, copy, cut, paste, and send those commands to theoperating system, instead of reporting touch coordinates or gestures.

Proximity sensor bar 510 includes housing 511 and lenses 310, throughwhich light beams 401, shown in FIGS. 32 and 35-38, are projected intothe detection plane as explained hereinabove. Light beams reflected byan object inserted into the detection plane re-enter proximity sensorbar 510 through lenses 310.

Reference is made to FIGS. 32-36, which are simplified illustrations ofthe laptop accessory of FIGS. 30 and 31 situated along an edge of alaptop display to convert a non-touchscreen display into a touchscreendisplay, in accordance with an embodiment of the present invention.FIGS. 32-36 show proximity sensor bar 510 attached to the bottom edge oflaptop display screen 831. The detection plane detected by proximitysensor bar 510 is parallel to the surface of laptop display screen 831,as illustrated by light beams 401 projected out of proximity sensor bar510. Proximity sensor bar 510 is as long as the bottom edge of screen831 in order to provide touch sensitivity to the entire display.Different models of proximity sensor bar 510 are manufactured indifferent sizes to support different screen sizes. In some embodiments,housing 511 is magnetically attached to laptop 830 below the bottom edgeof screen 831. The detection plane is mapped to the screen surfaceaccording to the expected location of proximity sensor bar 510 relativeto screen 831. E.g., proximity sensor bar 510 may be placed along any ofthe four screen edges, each placement transposing the detection plane inrelation to the screen. In some embodiments, a socket is provided in thelaptop housing for receiving proximity sensor bar 510. In some cases, aconnector provided in the socket, and corresponding connection padsprovided on proximity sensor bar 510, are used to connect proximitysensor bar 510 to laptop 830, instead of wire 833 and the USB socket andplug discussed hereinabove.

Reference is made to FIG. 37, which is a simplified illustration of thelaptop accessory of FIGS. 30 and 31 situated along an edge of a laptopdisplay and rotated away from the display to provide a detection planein the airspace between the display and the keyboard, in accordance withan embodiment of the present invention. FIG. 37 shows proximity sensorbar 510 rotated away from display screen 831 such that the projectedlight beams 401, and the corresponding detection plane, are directedinto the airspace between display 831 and keyboard 832. Thisconfiguration is useful for browsing photos and presentations: the useradvances forward and backward through the presentation by swiping hishand or finger through the air across the detection plane, withouttouching the screen. Angle θ shown in FIG. 37 is 45°, but can be anyangle. In some embodiments, proximity sensor bar 510 reports the samecoordinates, gestures or commands, to laptop 830 regardless of whetherthe detection plane is parallel to the screen surface or directed intothe airspace away from the display. In other embodiments, proximitysensor bar 510 reports different coordinates, gestures or commands, whenthe detection plane is parallel to the screen surface and when it isdirected into the airspace away from the display. In other embodiments,when the detection plane is directed into the airspace away from thedisplay, proximity sensor bar 510 employs a subset of the coordinates,gestures or commands it employs when the detection plane is parallelscreen surface. E.g., relative movement gestures, such as sweep gesturesand pinch gestures, are supported, but the location of a detected objectis not mapped to a specific screen location when the detection plane isdirected into the airspace away from the display.

Reference is made to FIG. 38, which is a simplified illustration of thelaptop accessory of FIGS. 30 and 31 situated along an edge of a laptopdisplay and rotated away from the display to provide a detection planealong the surface of the laptop keyboard, in accordance with anembodiment of the present invention. FIG. 38 shows proximity sensor bar510 rotated away from display screen 831 such that the projected lightbeams 401, and the corresponding detection plane, are parallel to thesurface of keyboard 832. This configuration provides touch functionalityto the upper surface of keyboard 832. In this configuration, whenproximity sensor bar 510 is configured to provide mouse or trackpadfunctionality by reporting detected object coordinates as a singlefinger trackpad, laptop 830 can eliminate the trackpad typicallyprovided below the keyboard on laptops today. Rather, the user can usethe upper surface of the keypad as a trackpad. In some embodiments, inorder to enable both keyboard input and trackpad input, trackpadfunctionality is suspended when a key is depressed, and trackpadfunctionality is activated when a finger moves across the keys withoutdepressing any key at the end of the movement. Angle θ shown in FIG. 38is greater than 90°.

Some laptops, known as 2-in-1 laptops, may be configured in both laptopmode, having a keyboard in front of the display, and in tablet mode,having nothing in front of the display. When in tablet mode, proximitysensor bar 510 can be placed along the bottom edge of the display screenfacing away from the display; e.g., such that the detection plane isparallel to the tabletop on which the display is standing. The user canthen control the presentation or video on the display using gesturesperformed on the table surface. E.g., swipe along the table surfaceparallel to proximity sensor bar 510 to advance or go backward; pinch onthe table surface to zoom, perform a multi-finger rotation gesture onthe table surface to rotate an image on the display.

In some embodiments, the light emitters in proximity sensor bar 510 aresemiconductor laser diodes such as vertical-cavity surface-emittinglasers (VCSELs). Other light emitters can alternatively be used. In someembodiments, proximity sensor bar 510 is manufactured by placinguncovered semiconductor laser diodes, i.e., the bare semiconductorwithout any lens, and uncovered photodiodes, also without any lens, ontoa PCB. The only lenses provided for the laser diodes and photodiodes isa light guide unit, such as the elongated light guide illustrated inFIG. 10, that includes lenses 303 and 304. The light guide unit ispositioned with very high precision relative to the VCSEL diodes by anautomated production line for high production volumes.

In the prior art optical components are aligned in an automatedproduction line by matching a hole pattern on the component carrier(PCB) with guides (pins) on the component to be placed. Alternatively,fiducial markers on the PCB are used to place the component accordingthe PCB patterns.

In contrast, the present invention uses the diodes themselves asfiducial markers to place the light guide exactly where it needs to bein relation to the diodes.

In some embodiments, prior to mounting the diode components, an adhesiveis attached to the PCB, which can be activated quickly; e.g. by exposureto UV light, to fix the component before the automated picking unitreleases it. Thus, the component is secured and fixed at its location onthe PCB before the light guide is mounted on the PCB. The light guide isthen picked up by the automated production line and positioned on thePCB by vision technology using the secured diodes as fiducial markers,thereby placing the light guide on the PCB in precise relation to thediodes. This precise positioning methodology increases the opportunityfor advanced high resolution applications at a competitive cost.

The elongated light guide illustrated in FIG. 10 includes lenses 303 and304, and more generally, includes multiple lenses that correspond torespective emitters and light detectors. In some embodiments, each lensis assembled on the PCB separately, in relation to its correspondingemitter, using that emitter as a fiducial marker.

Reference is made to FIG. 39, which is a simplified flow chart of aprocess for assembling a proximity sensor, in accordance with anembodiment of the present invention. The process uses a robot thatfetches optical components and places them on a PCB using machinevision. At step 1007, the robot applies glue to the PCB at a location atwhich an optical component such as a VCSEL semiconductor diode or asemiconductor photodiode is to be placed. At step 1008, the robot placesthe optical component on the glue, and at step 1009, the glue ishardened by exposure to UV light without the robot releasing the opticalcomponent. At step 1010, the robot fetches a lens for the glued opticalcomponent. Because the optical component is affixed to the PCB withoutany package, the small size of the components facilitates its use as afiducial marker for placement of the lens. Thus, the robot aligns thelens on the PCB with that lens's optical component using the opticalcomponent as a fiducial marker for placement of the lens. At step 1011the robot advances to repeat steps 1007-1010 for the next opticalcomponent and its corresponding lens.

Reference is made to FIG. 40, which is a simplified illustration of aproximity sensor being assembled according to the process of FIG. 39. Ateach step, one emitter and one lens are assembled on the PCB. FIG. 40shows proximity sensor PCB 512 after six emitter-lens pairs have beenmounted thereon. Each emitter 111-116 is placed on PCB 512 by anautomated picking unit and attached thereto, e.g., by exposure to UVlight as discussed hereinabove. Next, the automated picking unit fetchesa respective lens 311-316 and mounts it on PCB 512 using thecorresponding emitter as a fiducial marker. This precise placement isillustrated by the enlarged inset showing alignment of emitter 111 withlens 311. This processed is then repeated for the remaining emitters andtheir respective lenses, assembling an entire proximity sensor bar.

Reference is made to FIGS. 41 and 42, which are simplified illustrationsof light beams of a proximity sensor detecting an object, in accordancewith an embodiment of the present invention. FIGS. 41 and 42 show alight path used to detect an object, and individual lens structures321-325. Each lens structure serves a respective opposite emitter andtwo detectors, one to the left of the emitter and one to the right ofthe emitter. Thus, for example, lens structure 325 serves emitter 105and detectors 205 and 206. In addition each detector is served by twolens structures; e.g., detector 205 receives reflected light from lensstructures 324 and 325. In the example shown in FIGS. 41 and 42, lightfrom emitter 105 is reflected by an object (not shown) into lensstructure 323 and onto detector 203. Three segments of the detectedlight are indicated in FIGS. 41 and 42; namely, light beam 412 projectedoutward from lens structure 325 and radially outward of the proximitysensor, light beam 413 reflected by the object into lens structure 323,and light beam 414 directed by lens structure 323 onto detector 203.

Reference is made to FIG. 43, which is a simplified illustration of aside view of a proximity sensor and light beams projected therefrom, inaccordance with an embodiment of the present invention. FIG. 43 showslight beams projected radially outward from a proximity sensor, and acutaway side view of the light path illustrated in FIGS. 41 and 42.Light beam 411 from emitter 105 enters lens structure 325, where it isredirected outward as light beam 412.

Reference is made to FIG. 44, which is a simplified illustration of aproximity sensor lens and associated optical components viewed fromabove and light beams projected through that lens, in accordance with anembodiment of the present invention. FIG. 44 shows a top view of lens325 configured both for collimating light rays 411 from emitter 105 andfor collecting light rays 414 onto PD 205. Different portions ofentry/exit surface 326 are optimized for collimating light rays 411 fromemitter 105 entering through surface 326 and for collecting rays 414exiting through surface 326 onto PD 205. Indeed, that portion of surface326 opposite emitter 105 is convex, whereas the portion of surface 326optimized for collecting light onto PD 205 is concave. Thus, surface 326is alternately concave and convex. However, the remaining surfaces inlight guide 325 serve both the incoming and outgoing light beams 411 and414; only at surface 326 are the two sets of light beams refracted bydifferent surfaces.

Reference is made to FIG. 45, which is a simplified illustration of aside view of lens 325 and components of FIG. 44 and light beamsprojected through that lens, in accordance with an embodiment of thepresent invention. FIG. 45 shows lens 325 collimating emitter beams 411and concentrating incoming beams 414 onto PD 205. Lens 325 has a foldedlens structure providing two, internal collimating reflective surfaces.The folded lens structure is discussed in U.S. Pat. No. 9,063,614entitled OPTICAL TOUCH SCREENS and incorporated herein in its entiretyby reference. The lenses described hereinabove with reference to FIGS.41-45 are usable for various proximity sensors, inter alia, for theproximity sensors described hereinabove with reference to FIGS. 1-39,for a touch-sensitive steering wheel as discussed in U.S. Pat. No.8,775,023 entitled LIGHT-BASED TOUCH CONTROLS ON A STEERING WHEEL ANDDASHBOARD and incorporated herein in its entirety by reference, and forproximity sensors for car doors as discussed in U.S. Publication No.2015/0248796 A1 entitled DOOR HANDLE WITH OPTICAL PROXIMITY SENSORS andincorporated herein in its entirety by reference.

Reference is made to FIG. 46, which is a simplified illustrations of anL-shaped optical proximity sensor situated at a corner of a screen, inaccordance with an embodiment of the present invention. The proximitysensor employed in this embodiment is any of the proximity sensorsdiscussed hereinabove. The embodiment shown in FIG. 46 provides mousetracking or trackpad functionality on a display device, inter alia, acomputer monitor, a television, a tablet computer or a laptop computer,in case there is no space to fit a normal trackpad, or in case the userdoes not want to touch a surface for various reasons, or as a very cheapalternative for providing touch sensitivity to a portion of a screen.FIG. 46 illustrates a corner of a display monitor, and an enlarged viewof the touch sensitive portion. Optical proximity sensor 519 is formedinto an “L”—shape and is situated along a short segment of adjacentedges of screen 831. Optical proximity sensor 519 projects light beamsperpendicular to the screen surface, i.e., towards the user facing thescreen, as illustrated by detection planes 971 and 972, and therebytracks movements in the X and Y directions in the airspace above thetwo-dimensional portion of screen 831 bordered by optical proximitysensor 519. This airspace operates as a virtual trackpad wherebymovements in the X and Y directions in this airspace control movement ofa cursor on the screen, or manipulate a displayed image, e.g., scrolland zoom. In some embodiments, optical proximity sensor 519 projectslight beams at a non-perpendicular angle to the screen surface.

In some embodiments of the present invention movement of a hand in theairspace above sensor 519 is tracked by the sensor based on thenon-uniform surface of the hand being tracked, e.g., the palm orfingers. For example, when tracking the palm, different parts of thepalm surface reflect the projected light differently, which enables thesensor to identify a direction of movement of the different parts of thepalm and combine those movements into a single directional gesture.

Reference is made to FIG. 47, which is a simplified flow diagram of amethod of identifying a gesture, in accordance with an embodiment of thepresent invention. The method of FIG. 47 calculates a directionalmovement of an object in accordance with an embodiment of the invention.At each sampling time t, proximity sensor 519 identifies a plurality oflocal maxima among the co-activated emitter-detector pairs. FIG. 47shows a plurality of local maxima, a, b through n, each maximum beingtracked at times, t₁, t₂ through t_(m). Proximity sensor 519 determinessimultaneous movements of each local maximum, a, b through nrespectively, over a time interval. At the end of the time interval,proximity sensor 519 combines these simultaneous tracked movements intoa single global directional movement and interprets that globaldirectional movement as a user input gesture. Although reference is madeto proximity sensor 519 performing these operations, various operationsof the process may be performed by a separate processor while stillfalling within the scope of the present invention. Also, reflectionvalues other than maxima may be tracked, e.g., a group of neighboringreflection values forming a pattern may be tracked instead of anindividual maximum value. A plurality of these tracked patterns can becombined to identify a global movement. Alternatively, a tracked patternof neighboring reflection values is used as the identified globalmovement of the object.

In addition to using two-dimensional sweep gestures above the screen tomanipulate a cursor or image in two dimensions, the user also moves thecursor or image along the x-axis by sliding his finger along the X-axisportion of L-shaped proximity sensor 519. Similarly, the user moves thecursor or image along the y-axis by sliding his finger along the Y-axisportion of L-shaped proximity sensor 519. To select an item on thescreen at the cursor location, the user taps proximity sensor 519 at anylocation on proximity sensor 519; the tap need not be performed at thepreviously touched location.

Reference is made to FIG. 48, which is a simplified illustration of auser interface, in accordance with an embodiment of the presentinvention. FIG. 48 shows a user interface that displays a plurality oficons or buttons 975-977 wherein one icon or button is selected, asillustrated by hatching. The user's input changes which icon isselected, e.g., moving the hatching from icon 975 to icon 976, but nocursor image is provided. In this case, gestures described above formoving the cursor move the selection from one icon or button to another.Thus in FIG. 48 a left-to-right sweep gesture is detected by proximitysensor 519 and in response thereto selection moves from icon 975 to icon976. As discussed hereinabove, sensor 519 also detects two-dimensionalgestures such as diagonal sweep gestures and further detects approachgestures based on whether the object such as the user's hand is movingtoward the screen or moving away from the screen. Thus, gestures inthree dimensions are detected.

Reference is made to FIG. 49, which is a simplified illustration of anoptical proximity sensor situated along a short segment of a displayedge for adjusting display parameters, in accordance with an embodimentof the present invention. The proximity sensor employed in thisembodiment is any of the proximity sensors discussed hereinabove. FIG.49 shows a monitor 831 having proximity sensor bar 520 situated along ashort segment of the bottom edge of monitor 831. The light beams fromproximity sensor bar 520 are directed parallel to the screen surface toprovide detection area 973, whereas the remainder of screen 831 formsnon-touch sensitive portion 974. Controls 981 and 982 for adjustingmonitor parameters such as brightness and contrast, are provided onscreen 831 within detection area 973. For example, main menu 981 enablesthe user to select a parameter to adjust from among brightness,contrast, color and volume. In FIG. 49 the user taps menu option 983 toselect brightness. In response to this tap, main menu 981 is replacedwith slider control 982 in detection area 973 enabling the user to dragscroll knob 984 along the slider bar-right to increase brightness, andleft to decrease brightness.

Another user interface is a GUI for a display or HUD mounted in avehicle, described by FIGS. 50-58. The GUI provides two different modesof navigating through the GUI options: (i) contextual navigation throughapplication cards, and (ii) hierarchical navigation using nested levelsof menus and lists.

Reference is made to FIGS. 50, 53, 54 and 57, which are flow charts foran in-vehicle infotainment system GUI, in accordance with an embodimentof the present invention. Reference is also made to FIGS. 51, 55, 56 and58, which are screenshots of the in-vehicle infotainment system GUI ofFIGS. 50, 53, 54 and 57, in accordance with an embodiment of the presentinvention. As show in FIG. 50, at step 1020 a plurality of applicationcards representing frequently used media, phone and navigation and othercontextual events relevant to the user are arranged on the display. Insome embodiments these cards are arranged in a row that extends beyondthe display, and the user pans the row of cards to move cards into andout of the display, as indicated by steps 1021-1024. The user taps anapplication card to select it as the active application in the GUI.

FIG. 51 shows application cards 986-988 arranged in a horizontal rowacross display 831.

Returning back to FIG. 50, at steps 1025-1027 a notification ispresented by the GUI as a card inserted in the middle of the displaywithin the row of application cards, moving other cards in the row leftor right. Reference is made to FIG. 52, which is an illustration of anotification card used in the in-vehicle infotainment system GUI ofFIGS. 50, 51 and 53-58, in accordance with an embodiment of the presentinvention. FIG. 52 shows a notification card 985 for an incoming call.To dismiss the notification, e.g., to reject the incoming call, the userswipes upward on the display, as per steps 1041-1043 in FIG. 53. Toaccept the notification, e.g., to accept the incoming call, the usertaps on the display as per steps 1044 and 1045 In FIG. 53.

Returning back to FIG. 50, at steps 1028-1030, a list of contextualoptions for the currently active application card is displayed inresponse to the user approaching the display, e.g., by extending a handtoward the display. List items 900-902 in FIG. 51 illustrate such acontextual list for current active application 987.

FIG. 54 shows the behavior once a contextual list is opened in the GUI.The user can still navigate through the row of contextual applicationcards by swiping left and right, as per steps 1052-1055. At step 1051,during such navigation, the list of contextual options changes accordingto which application card is at the center of the display. At steps 1056and 1057, a tap on a list item activates that item.

Returning back to FIG. 50, at steps 1031-1033, the GUI provideshierarchical navigation using nested levels of menus and lists inresponse to a tap at the bottom of the display. When this occurs, bothcontextual navigation through application cards, and hierarchicalnavigation using nested levels of menus and lists, are provided at thesame time. Thus, the user accesses and activates any of the applicationsthrough the application cards, and also activates those sameapplications though a set of hierarchical menus.

FIG. 55 shows primary menu bar 903, including category items 904-908,opened in response to a tap at the bottom of display 831. A tap on anycategory item 904-908 opens a secondary menu bar within display 831,presenting subcategories within the selected category, and dividing thedisplay into an upper portion and a lower portion.

FIG. 56 shows primary menu bar 903, and secondary menu bar 996,including secondary categories 997-999, related to selected primarycategory 906. Contextual application cards 986-988 are reduced in sizeabove secondary menu bar 996, and can still be navigated and selected asbefore. This is indicated at steps 1080-1087 in FIG. 57. A tap on anysecondary category in secondary menu bar 996, opens list of relevantoptions, inter alia 978-980, below secondary menu bar 996, as shown inFIG. 56 and as indicated at steps 1075 and 1076 in FIG. 57. A tap on anylist item activates that item, as described by steps 1077 and 1078 inFIG. 57.

FIG. 58 shows an incoming notification when both the contextualnavigation through application cards, and hierarchical navigation usingnested levels of menus and lists, are provided at the same time.Notification card 985 is inserted within the application cards abovesecondary menu bar 996. To dismiss the notification, e.g., to reject theincoming call, the user swipes upward above secondary menu bar 996. Toaccept the notification, e.g., to accept the incoming call, the usertaps on notification 985.

In the foregoing specification, the invention has been described withreference to specific exemplary embodiments thereof. It will, however,be evident that various modifications and changes may be made to thespecific exemplary embodiments without departing from the broader spiritand scope of the invention. Accordingly, the specification and drawingsare to be regarded in an illustrative rather than a restrictive sense.

1-20. (canceled)
 21. A proximity sensor for detecting presence ofobjects in three-dimensional space, comprising: a housing; a PCB in saidhousing; a plurality of light emitters, denoted E, mounted on said PCBin a curved arrangement, each light emitter projecting a directed lightbeam out of said housing at a particular angle relative to the plane ofthe PCB; a plurality of light detectors, denoted E, mounted on said PCBand interleaved with said light emitters in the curved arrangement; aplurality of lenses, each lens oriented relative to a respective one ofsaid light detectors in such a manner that the light detector receivesmaximum intensity when light enters the lens at a particular angle,denoted b, whereby, for each emitter E_(i), there exists a positiveinteger j, and a plurality of corresponding target positions p(E_(i),D_(i,1)), . . . , p(E_(i), D_(i,j)) along the path of the directed lightbeam from emitter E_(i), at which an object located at any of the targetpositions will reflect the light projected by emitter E_(i) towards arespective one of detectors D_(i,1), . . . , D_(i,j) at angle b, whereinall target positions p(E, D) are located in a 3D volume above saidhousing; and a processor connected to said emitters and to saiddetectors, operable (i) to synchronously activate each emitter with oneor more of said detectors, each synchronously activated emitter, E, anddetector, D, denoted an emitter-detector pair (E, D), (ii) to store areflection value R(E, D) for each synchronously activatedemitter-detector pair (E, D), based on an amount of light reflected byan object located at target position p(E, D) and detected by detector Dwhen the pair (E, D) is synchronously activated, (iii) to associate thereflection value R(E, D) with the target position p(E, D), and (iv) tocalculate a location of an object in the 3D volume based on thereflection values R(E, D) and the target positions p(E, D).
 22. Theproximity sensor of claim 21, wherein said processor is furtherconfigured to generate a map of reflection values Rp at positions p inthe 3D volume, corresponding to the derived reflection values R(E, D)and the target positions p(E, D).
 23. The proximity sensor of claim 22,wherein said processor is further configured to identify a partialcontour of the object in the 3D volume based on the generated map ofreflection values.
 24. The proximity sensor of claim 21, wherein saidprocessor is further configured to detect in-air gestures performed bythe object in the 3D volume based on the reflection values R(E, D),generated over a time interval, and the target positions p(E, D). 25.The proximity sensor of claim 21, wherein said processor is furtherconfigured to identify a size of the object in the 3D volume based onthe reflection values R(E, D) and the target positions p(E, D).
 26. Theproximity sensor of claim 21, wherein the curved arrangement iscircular.
 27. The proximity sensor of claim 21, wherein the curvedarrangement is wave-shaped.
 28. A method for sensing an object withinthree-dimensional space, comprising: arranging a plurality of lightemitters E and light detectors D along a planar curve; arranging aplurality of lenses, such that light from each light emitter E isdirected by a respective lens as a beam at a particular angle relativeto the plane of the curve, and each light detector D receives maximumintensity when light enters a respective lens at a particular angle,denoted b, whereby, for each emitter E_(i), there exists a positiveinteger j, and a plurality of corresponding target positions p(E_(i),D_(i,1)), . . . , p(E_(i), D_(i,j)) along the path of the directed lightbeam from emitter E_(i), at which an object will reflect the lightprojected by emitter E_(i) towards a respective one of detectorsD_(i,1), . . . , D_(i,j) at angle b, wherein all target positions p(E,D) are located in a 3D volume; synchronously activating each emitterwith one or more of the detectors, each synchronously activated emitter,E, and detector, D, denoted an emitter-detector pair (E, D); determininga reflection value R(E, D) for each emitter-detector pair (E, D), basedon an amount of light reflected by the object located at target positionp(E, D) and detected by detector D, and associating the reflection valueR(E, D) with the target position p(E, D); and detecting an object basedon the reflection values R(E, D) and the target positions p(E, D). 29.The method of claim 28, further comprising, generating a map ofreflection values Rp at positions p in the 3D volume, corresponding tothe derived reflection values R(E, D) and the target positions p(E, D).30. The method of claim 29, further comprising identifying a partialcontour of the object in the 3D volume based on the generated map ofreflection values.
 31. The method of claim 28, further comprising,identifying in-air gestures performed by the object in the 3D volumebased on the reflection values R(E, D), generated over a time interval,and the target positions p(E, D).
 32. The method of claim 28, furthercomprising identifying a size of the object in the 3D volume based onthe reflection values R(E, D) and the target positions p(E, D).
 33. Themethod of claim 28, wherein the plurality of light emitters and lightdetectors is arranged in a circle.
 34. The method of claim 28, whereinthe plurality of light emitters and light detectors is arranged along awave-shaped curve.